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What arc Performance Indicators and Can they Be Used to Manage Colleges? 



A Comparative Study of similar Colleges in the US and the UK 



Abstract 



Performance indicators, student outcomes assessment, quality indicators, institutional effectiveness 
indicators are all attempts to identify how well colleges do what they say they do. In an attempt to think 
outside of our parochial boxes we have chosen to examine our two similar institutions (in different 
locations) regarding indicators used to manage our colleges. 

The institutions in this study are both Technical Community Colleges in industrialized cities in the US 
and the UK with significant minority enrollment. Both Wisconsin and England have been identified by 
Olaf C. McDaniel as having decentralized governance (Paradigms of Governance in Higher Education, 
Vol. 9, June 1996). 

We have addressed the following issues; how we measure ourselves, how we compare to each other, and 
what we can learn from each other. We are including in our discussion the concept of "Value Added to 
the Community" in response to the Community College research agenda set forth by the AACC 
(American Association of Community Colleges). 
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What arc Performance Indicators and Can they Be Used to Manage Colleges? 
A Comparative Study of similar Colleges in the US and the UK 



Introduction 

" Not everything that can be counted counts; and not everything that counts can be counted. 

- A Ibert Einstein 



This paper will consider performance indicators that measure how effectively two inner city Community 
Colleges in different countries are providing ‘Value Added” to their local regional and national 
communities. ‘Value Added to the Community’ is one of the specified areas within the American 
Association of Community Colleges’ Research Agenda for 1996/97 and opportunely fits within existing 
research areas that both colleges have been pursuing for some time. 

Each college sees performance indicators within the context of the greater environment. In the US 
example, Milwaukee Area Technical College, the influencing factors are the Institutional Effectiveness 
Measures developed for the Wisconsin Technical College System and the Student Outcomes Measures 
expected by the North Central Association accrediting agency. The UK example, Handsworth College, is 
influenced by the expectations of the Further Education Funding Council. 

In several states in the United States the idea of performance indicators has been the result of legislative 
action and tied somewhat to funding. South Carolina, for example, is currently in the process to tying all 
its funding to performance indicators. This has not been the case in Wisconsin. In late 1992 the 
Wisconsin Technical College Board, along with the State Technical College Presidents Association, 
proposed the development of a model for institutional effectiveness for the Wisconsin technical colleges. 
The Community College Consortium at the University of Michigan coordinated the efforts. The main 
objective was to “develop a comprehensive effectiveness model for adaptation within the context of the 
individual colleges.” A task force was assembled with representatives of the member colleges of the 
Wisconsin Technical College System working with many constituent groups to develop the core indicators 
and a definition of effectiveness. 

Effectiveness was defined as: 

“An effective college is one that provides efficient educational programs and services that anticipate and 
respond to external and internal customer needs and result in outcomes that meet or exceed customer 
goals and expectations” 

The task force developed seventeen core indicators of effectiveness. They are: 

Indicators of Employer Satisfaction 

WI 1 Employer satisfaction with graduates’ work performance. 

Indicators of Student Achievement and Satisfaction 

WI 2 Identification of student needs, goals and interests. 

WI 3 Identification of student functional skills at entry. 

WI 4 Course completion. 

WI 5 Student grades. 

WI 6 Student satisfaction with courses, programs and services. 

WI 7 Student retention rates. 

WI 8 Student completion and graduation rates. 
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WI 9 Student achievement of educational goal(s). 

WI 10 Student knowledge and skills at exit. 

Wl 1 1 Pass rates/scores on licensure exams. 

WI 12 Placement rates/employment rates. 

Indicators of Organizational Quality, Harmony and Efficiency 
WI 13 Achievement of institutional goals and standards. 

WI 14 Organizational climate. 

Indicators of Public Satisfaction 

WI 15 Articulation and linkages with external organizations. 

WI 16 Identification of customer needs and expectations. 

WI 17 Public satisfaction. 

Additional performance indicators used in Wisconsin relate to the Carl Perkins funds for vocational 
education. These moneys are targeted to “special populations” of people who are disabled, academically 
disadvantaged, economically disadvantaged, limited English speaking, plus programs to eliminate sex 
bias and persons in correctional institutions. These standards have to do with the performance of the 
“special populations” relative to the total student population. Therefore, statewide standards have been 
established. These measures and standards are as follows; 

Measure 1 : Program Graduates 
Standard 1: 40% 

Measure 2: Employment and Availability for Work 
Standard 2; 85% 

Measure 3: Number and Percent of Population Served Who had a Least One Training Outcome. 
Standard 3: 50% 

Measure 4: Number and Percent of Population Served 
Standard 4: Set locally 

Measure 5: Successfully Completed Courses 
Standard 5: Set locally 



In the UK, on the other hand, all the funding is tied to performance indicators. The Further Education 
Funding Council (FEFC) is an arm of government which provides the large majority of the funding for 
colleges such as Handsworth. The FEFC has identified a range of performance indicators and 
management statistics which it calculates from data provided in statutory returns by the colleges. This 
information is published annually for each college and is commonly known as ‘The League Tables’ where 
the performances of different colleges can readily be compared with each other. 

The performance indicators defined in this way are: 

PI 1 Achievement of the volume of education on which FEFC income is based 

PI 2 Student Number trends - an indicator of college responsiveness 

PI 3 Student continuation rates (retention) 

PI 4 Student achievement rates 

PI 5 Numbers of students achieving qualifications towards National Training and Education Targets 
(NTETs) 

PI 6 Out-turn level of funding (value for money) 
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The management indicators are: 

MS 1 Students by age group and sex 

MS 2 Students by qualification (Analysis of what the students are studying) 

MS 3 Fee Remission (Analysis of reasons why students have free tuition fees) 

MS 4 Student Retention 

MS 5 Staff numbers by age and sex 

MS 6 Staff FTEs by type of work and contract 

MS 7 Current assets to current liabilities 

MS 8 Liquid assets to current liabilities 

MS 9 Cash days in hand 

MS 10 Annual surplus as percentage of income 

MS 1 1 General reserves as a percentage of income 

MS 12 Long-term liabilities as a percentage of reserves 

MS 13 Cash generated from operations as a percentage of income 

MS 14 Pay expenditure as a percentage of income 

MS 15 New building costs 

MS 16 Space efficiency 

MS 17 Value of college estates 

MS 18 Total expenditure per student FTE 

MS 19 Pay expenditure per student FTE 

MS 20 Pay expenditure on teaching and leaning per student FTE 

MS 2 1 Council Funding per achieving student FTE 

MS 22 Total expenditure on teaching support services per student FTE 

MS 23 Student FTEs per staff FTE spent on providing or supporting teaching and learning 



FE colleges are inspected officially every three to four years by inspectors appointed by the FEFC. For 
each college, gradings together with a lengthy commentary are published nationally. These gradings are 

IN 1 Responsiveness and range of provision 
IN 2 Governance and management 
IN 3 Students’ recruitment, guidance and support 
IN 4 Quality Assurance 

IN 5 Resources for each of staff, equipment/leaming resources and accommodation 
IN 6 The quality of the teaching in each of the curriculum areas 



In addition to these the college regularly calculates other indicators, such as; 

CL 1 Value added - comparison of student’s abilities and previous education compared with 
qualifications achieved at the end of the student’s chosen program. 

CL 2 Regular comparison of numbers of students studying and volume of education delivered against 
college plan. 

CL 3 Students’ progression rates to further and higher education, and employment. 
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Profile of Our Communities 



Both communities are located in large cities, one in the United States and the other in the United 
Kingdom. Both cities have traditional manufacturing industrial bases which have experienced decline in 
the last twenty years or so. The populations of both cities have substantial numbers of minority group 
members, many of whom are non-English speakers. 

Milwaukee 

The city of Milwaukee, population 623,531, is the 17th largest city in the United States. Milwaukee 
County’s population is nearly one million and the four-county metropolitan area has over 1.4 million 
people. There are about 38,000 metro Milwaukee businesses which employ over 802,600 people. This 
includes 13,700 service firms, 8,200 retailers, 3,200 manufacturers, 3,300 wholesalers and 3,500 
construction firms. Twenty-two percent of Milwaukee’s workforce is in manufacturing. 

Contrary to popular perceptions, less than 1% of Milwaukee’s workforce is employed in brewing. 

(Though many more are engaged in product testing, particularly when the Packers win the Superbowl!) 
Milwaukee area manufacturers are leaders nationally in the production of X-ray apparatus & tubes, small 
gasoline engines, malt beverages, iron & steel forgings, mining machinery, speed changers and drives, 
leather goods and lithographic commercial printing. Milwaukee is a leading producer of medical 
diagnostic instruments, computers, industrial robots, electronic controls & systems for factory automation, 
printed circuit boards, CAD/CAM processing, avionics and advanced ceramics. 

The African American population represents the largest minority group in metro Milwaukee. They 
numbered 197,183 in the 1990 census (13.8% of the population) Other minority groups were as follows; 
Hispanic at 5 1,306 (3.6% of the population), Asian at 1.3% and Native American at 0.6%. Of the 
European Americans in metro Milwaukee, 40% are of German heritage. 

The rate of unemployment in January 1997 in Milwaukee County is 4.0%. This compares with 5.5% in 
the City of Milwaukee, 4.3% in the State of Wisconsin and 5.9% in the US as a whole. As of February of 
1997, the unemployment rate in Wisconsin had dropped to 3.5%, the lowest in twenty years. 



Birmingham 

Birmingham with a population of 961,041 is the second largest city in England. It is located in the 
conurbation of the West Midlands which has a total population of 2.5 million. 

The part of Birmingham in which Handsworth College is located is a mainly a poor working-class area 
where the population is predominately non-white. The 1991 census shows that largest group in this area 
is Indian/Pakistani/Bangladeshi at 33.9%, followed by the Black ethnic group at 23.6% and others 
(including Chinese) at 5. 1%. Within the White ethnic group, 5.0% were born in the Irish republic. 

In the local area, in October 1996, 23.4% of the economically active adults aged over 16 were 
unemployed. This compares with 12.8% in the city of Birmingham and 7.9% for the UK as a whole. Of 
the employed adults the biggest group (38.9%) are employed in services (including banking and finance), 
24.9% are employed in manufacturing and 19.0% in distribution and catering. 

In levels of education achieved, 0.9% of the adult population has a higher degree and another 9.3% has a 
Bachelor’s degree or higher education diploma. 



Profile of Our Colleges 



Both colleges are located in the inner areas of industrial cities and have large numbers of students 
studying subjects across a wide range of curriculum areas. Both colleges have substantial numbers of 
students learning English as a second language. In addition, much of the teaching takes place in the 
community in addition to the colleges’ campuses. Milwaukee Area Technical College has been identified 
in a recent survey as being a ‘major contributor’ to the Quality of life in the area. Handsworth College has 
pioneered schemes to facilitate groups to set up and deliver education activities for themselves. The 
college provides mentoring and administrative support and has obtained national funding to support these 
activities. 



Milwaukee Area Technical College 

The Milwaukee Area Technical College (MAT C) is a four campus urban community-technical college 
located in the metropolitan area including and immediately surrounding the City of Milwaukee. MATC is 
one of sixteen districts that comprise the Wisconsin Technical College System and accounts for between 
20 and 25% of the total state Technical College enrollments. 

Founded in 1912 as the Central Continuation School and modeled on the fortbildungschule in Munich, 
MATC has a long history of vocational and technical education. In addition to the regular post-secondary 
offerings, MATC operates an Adult High School that has been accredited since the 1920’s. It is also a 
principal provider of adult basic education (ABE) in the Milwaukee Area. Over 16,000 students enroll in 
ABE and English as a Second Language (ESL) in a typical year through a system of on-campus 
instruction, in-house training in industry and instruction at community based organizations. The college 
operates the two public Television stations in Milwaukee and is also the principal investor in two small 
business incubators (the Milwaukee Enterprise Centers). Graduates from the many associate degree and 
diploma programs have a placement rate consistently over 90%. 

During the 1995-96 school year 31,1 13 students were enrolled in post-secondary programs, 19,462 were 
enrolled in the Adult High School and ABE/ESL courses and an additional 20,645 were enrolled in 
continuing education. MATC has approximately 600 full-time faculty and another 1,200 part-time 
faculty. 

MATC’s North Campus, among other things, offers associate degrees in automotive technology under 
contract arrangements with both Ford and General Motors. 

Although MATC is well known internationally, as evidenced by the large number of foreign educators 
and business people who visit the college and also by a recent three part series shown on Korean 
Television showcasing MATC, the college itself has not been active in delivering education 
internationally. 

The District is unique, as a public institution, in the large number of citizens who are involved in 
planning and reviewing its educational programs. These citizen’s advisory committees are composed of 
business, industry, and labor representatives as w r ell as current and past students. Currently MATC has 94 
separate advisory committees with 1,054 citizen representatives. 

The Board of Trustees is also representative of the community. The nine members of the Board of 
Trustees are appointed by the elected presidents of the k-12 school boards within MATC’s service area. 
The Board not only represents business, industry and labor but is also constructed to represent the ethnic 
and gender makeup of the district. 

This kind of involvement over the years has created a situation in winch the college is able to respond to 
market needs. 
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Handsworth College 



Handsworth College is one of eight further education colleges within the city of Birmingham. It is a six- 
campus urban further education college located near the center of the city. It was founded in 1893 as 
Handsworth Technical College and originally provided technical and commercial training to many of the 
small manufacturing companies in the local area. 

In recent years, the college has cultivated links with many minority groups in the metropolitan area and 
beyond and has enabled groups to provide education to students who, due to cultural influences, were 
unlikely to enter mainstream post-school education. In particular, Handsworth has become a center of 
excellence in ESL provision, and now operates such programs nationwide. By means of this type of 
activity Handsworth College has experienced substantial growth in the mid 1990’s where the student 
enrollment numbers have trebled in the past four years. 

Handsworth College has maintained its strong links with the motor vehicle industry and operates the 
Birmingham Automotive Training Centre in which students receive instruction in all aspects of the motor 
industry, in particular in collaboration with local companies such as Land Rover and Jaguar. 

Handsworth College has developed links with colleges in other countries. It has pioneered an associate 
degree program with the City Colleges of Chicago, and has developed ESL programs with a network of 
colleges in USA, Canada, Netherlands and South Africa. 

During the 1995/96 school year 22,372 students were enrolled at Handsworth College, who amounted to 
6,492 full time equivalents. These students were in studying in curriculum areas as shown in table 1 
below. 



Table 1 Handsworth College - Student Population 1995/96 


Curriculum Area % of FTEs 


Ethnicity % of FTEs 


Science 5.9% 

Humanities 19.9% 

Engineering 13.2% 

Business Studies 12.7% 

Hotel and Catering 6. 1% 

Health & Community Care 19.7% 

Art and Design 9.2% 

Basic Education 14. 1% 


Black 17.1% 

Chinese/Asian 2.3% 

Indian/Pakistani 36.2% 

White 44.4% 



The College Corporation (Board of Trustees) comprises 17 members of whom 11 represent local business 
and industry and the remainder represent the local community. 



Performance Indicators 

Over the last several years there has been a wide variety of methods, terms and concepts used to address 
accountability. We have discussed Institutional Effectiveness, Student Outcomes Assessment, Program 
Viability, Faculty Productivity and various other topics to represent measures of accountability. The term 
that we feel is most appropriate for our discussion is “Performance Indicator” because we feel that all of 
these terms and concepts are really ways of indicating how the institution is performing. We also 
understand that performance indicators are just a part of a broader context in which post secondary 
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education is operating both in the United States and Europe. Martin Trow states that “...the underlying 
nature of accountability is one of three fundamental ways in which colleges and universities are linked to 
their surrounding and supporting societies: the others are trust and the market". 

We find it appropriate to define the place in post secondary education where each of our institutions are 
situated. In the UK, a clear distinction is made between “Higher Education' and “Further Education” 
and Handsworth college is clearly a further education institution. In the US no such clear distinction is 
made, and, in fact, community colleges and other two year colleges are usually thought of, discussed and 
accredited as institutions of Higher Education. But when comparing the two institutions, MATC is clearly 
the kind of institution that would be considered as a further education college in the UK. Therefore when 
we reference “further education” in this paper we will be referring to both institutions. 

In order to address Trow’s three fundamental linking activities in the proper context, the placement of our 
institutions within the mosaic of post secondary education in Europe and in the US was necessaiy. It is 
our position that older established universities have value in their communities because of the factor of 
trust. Further education institutions on the other hand have historically existed because of their apparent 
response to market needs. In both the US and the UK it is easy to develop a continuum of post secondary 
education with Old Prestigious Universities on one side (Oxford, Harvard), and further education colleges 
on the other (Handsworth College, Milwaukee Area Technical College). Although all post secondary 
education must attend to all of the three linking activities, it is clear to us that further education colleges 
can depend less upon trust and more upon accountability and markets. The American Association of 
Community Colleges identification of ‘Value Added to the Community” as a research need reinforces the 
interrelationship of accountability and markets for these colleges. For the purposes of our discussion, 
accountability will be measured by performance indicators. The inherent problem with accountability is 
one which is highlighted by Trow when he states “Education is a process pretending to have a measurable 
outcome.” This is especially true of the institutions depending largely upon trust. Those depending 
largely upon markets have a history of thinking that they have measurable outcomes and therefore are 
more likely to be involved in the “performance indicator” discussions. 



Aspects of Accountability 

The United States and the United Kingdom differ in the circumstances of their accountability to their 
respective communities. In the US regional accrediting bodies dictate the accountability measures 
whereas in the UK the accountability measures are administered by the government and linked closely to 
funding. In either case, there are certain aspects of accountability that have been discussed by Trow. 

These are first, distinction between external and internal accountability on one axis and a distinction 
between “legal and financial” accountability and “academic” accountability on the other axis. This can be 
illustrated by the 2X2 matrix below: 



Table 2 


Accountability Matrix 




Accountability 


Legal/financial 


Academics 


External 


Cell 1 


Cell 2 


Internal 


Cell 3 


Cell 4 
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Each performance indicator can be placed in one or more cells in the above matrix. The nature of the 
placement of the indicators will give the institution insights into what sources are driving the need for 
accountability in their institutions. The cell placement of the indicators identified for the Wisconsin 
Technical College System and for Handsworth college are shown in tables 3 and 4 below: 



Table 3: WISCONSIN ACCOUNTABILITY MATRIX 




LEGAL/FINANCIAL 


ACADEMIC 


EXTERNAL 


WI 7 


Student retention rates 


WI 1 


Employer satisfaction with 




WI 8 


Student completion and 




graduates’ work performance 






graduation rates 


WI 4 


Course completion 




WI 15 


Articulation and linkages with 


WI 6 


Student satisfaction with course. 






external organizations 




programs, and services 








WI 9 


Student achievement of 










educational goal(s) 








WI 10 


Student knowledge and skills at 










exit 








WI 11 


Pass rates/scores on licensure 










exams 








WI 12 


Placement rates/employment rates 








WI 15 


Articulation and linkages with 










external organizations 








WI 16 


Identification of customer needs 










and expectations 








WI 17 


Public satisfaction 






3 items 




10 Items 


INTERNAL 


WI 2 


Identification of student needs, 


WI 9 


Student achievement of 






goals, and interests 




educational goal(s) 




WI 3 


Identification of student 


WI 10 


Student knowledge and skills at 






functional skills at entry 




exit 




WI 5 


Student grades 


WI 13 


Achievement of institutional goals 




WI 13 


Achievement of institutional 




and standards 






goals and standards 


WI 14 


Organizational climate 




WI 14 


Organizational climate 










5 items 




4 items 



12 
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Table 4: HANDSWORTH COLLEGE ACCOUNTABILITY MATRIX 




LEGAL/FINANCIAL 


ACADEMIC 


EXTERNAL 


PI 1 Achievement of volume of 


PI 3 Student continuation rates 




education on which FEFC 


(retention) 




income is based 


PI 4 Student achievement rates 




PI 2 Student Number trends - an 


PI 5 Numbers of students achieving 




indicator of college 


NTETs 




responsiveness 


MS 2 Students by qualification (Analysis 




PI 6 Out-turn level of funding (value 


of what the students are studying) 




for money) 


MS 4 Student Retention 




MS 1 Students by age group and sex 


IN 1 Responsiveness 




MS 3 Fee Remission (Analysis of 


IN 3 Students’ recruitment, guidance 




reasons why students have free 


and support 




tuition fees) 


IN 4 Quality Assurance 




MS 5 Staff numbers by age and sex 






MS 6 Staff FTEs by type of work and 






contract 






MS 7- 17 other financial indicators 






MS 23 






26 items 


8 items 


INTERNAL 


PI 2 Student number trends - an 


CL 1 Value added - comparison of 




indicator of college 


student’s abilities/previous 




responsiveness 


education compared with 




CL 2 Regular comparison of numbers 


qualifications achieved at the end 




of students studying and volume 


of the student’s chosen program 




of education delivered against 


CL 3 Students’ progression rates to 




college plan 


further programs and employment 




8 items 


2 items 



Each indicator is placed in at most two cells by the authors based upon their perceptions of the 
motivations the indicators together with count of the number of indicators in each cell. 

The tables above illustrate that in both instances the driving forces of accountability are external. In the 
case of MATC, the majority of the external forces are pressures that are forcing academic accountability. 
In Handsworth’s case the majority of the external forces are legal or financial. This is consistent with the 
historical source of the measures. In the case of the Wisconsin indicators, they were developed by 
representatives of the member colleges in response to what looked like political pressure from outside of 
the colleges. In England, on the other hand, the indicators were bureaucratically imposed for purposes of 
equitable funding. 
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Relationship to Funding 



The accountability measures in the UK are tied closely to funding while the accountability measures in the 
US have little or no relationship to funding. We will start with the Wisconsin example. The State 
Technical College Funding formula accounts for approximately 22% of the budget, local property tax 
accounts for 53%, tuition and fees account for 14%, federal 5% and other 6%. None of these funds are in 
response to any performance indicators that are usually defined in the literature and used in college 
planning and accreditation. The only indicator that has any relationship to funding is number enrolled, 
or, more precisely, FTE. 

Let us examine the technical college state aid formula in Wisconsin. It is essentially cost driven. That 
means that the amount that a local technical college district spends determines how much it receives in 
state aid. Each local district has an annual cost of operating the district with certain inclusions and 
exclusions by state statute. This cost is the “aidable cost” for that district. This factor is then modified by 
an “equalization index” (El) which is defined by the formula below: 

El = (Statewide Property Value)(District Aidable FTE)/(Statewide Aidable FTE)(District Property Value) 

The “equalization index” puts all the technical college districts in competition with each other for the 
fixed State funds available. Examining the El formula shows that not only are a college’s district 
property assessed valuation and FTEs important but they are important relative to the property values and 
FTEs in the entire state. Trying to map this funding formula back to any kind of performance indicator is 
fruitless. 

In the case of Handsworth College, the FEFC provides 86% of the budget, the European Union 3%, 
tuition fees 1% and 10% comes from a range of sources such as other government departments. 

The method of funding further education in the UK w'as revolutionized in 1994, when the FEFC was 
established and introduced a formula based funding methodology to apply to all colleges in the sector. 

The aims of the method were to be fair and equitable to all, to be easy to understand, to maintain funding 
stability and to produce efficiency gains in a growing sector. In practice, the methodology has become 
increasingly complex as new features have been added to the formula to endeavor to maintain fairness to 
colleges and at the same time to ‘encourage’ compliance to government initiatives. 

The method basically operates as follows. Some four months prior to the start of the school year, the 
‘volume of education’ (known as Funding Units) that each college is to be contracted to deliver, is 
determined by the FEFC based on college bids and other criteria that the FEFC applies. In practice, ever} 7 
college is allocated a number of units based on the previous year, together with a level of growth. The 
units are multiplied by the ‘Average Level of Funding’ which is a calculated amount of money for each 
unit. This gives a budget figure on which the college is funded. For instance, in 1996/97 Handswurth 
College has been funded to deliver 912,000 funding units @ £14.32 per unit, giving a budget income of 
£13. lm. 

The Average Level of Funding (ALF) is based on the college’s previous value and a convergence factor. 
Currently, Handsw'orth College has the lowest budgeted ALF in England where its value is about half that 
of the highest in general further education. The FEFC is seeking to ensure full convergence will be 
achieved within the next three to five years whereafter the ALF will be the same for all colleges. This is 
likely to cause financial problems for the high ALF colleges as their budgets are reduced. 

At three census dates during the school year, the college’s progress towards its target number of funding 
units is calculated and money is ‘clawed back’ if it is not close to target. Until recently, there w^ere funds 
available for colleges who exceeded the target, but this has now been stopped. The funding units for each 
student are based on a complex formula dependent on the nature of the student’s program, the student’s 
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retention and the student's achievement of the qualification. The biggest factor is student retention as 
units are deducted proportionally based on the dates students withdraw. 

The whole process is subject to intense audit scrutiny. Teams of professional auditors visit the college 
several times a year to check the records and report on their accuracy. A whole new industry has now 
been created in running training events on how to optimize the methodology and then afterwards survive 
the audit. 



The two funding formulae are both examples of bureaucratic attempts to be equitable in the distribution of 
funds for the respective colleges. The Wisconsin formula attempts to dispense funds in such a way that 
the districts that have fewer potential resources (property tax base) are compensated by the State so that 
those districts can offer programs at an equal quality level to the more affluent districts. The relative 
performance of the individual districts is accounted for in the Wisconsin formula by the FTE factors. 

In England, on the other hand, equitable distribution means funding allocation relative to a unit of 
production. Funds are allocated according to performance measures that describe that production. 



Value Added to the Community 

The American Association of Community Colleges has placed “Value added to the community” on it’s 
research agenda for 1996-97. This fits our discussion well because it addresses the two sectors in which 
our colleges interact with the community: markets and accountability. By nature, further education 
colleges and community colleges are intended to be responsive to the community and therefore adding 
value to that community. 

The Milwaukee Area Technical College has been fortunate in that a local survey research firm, 
Management Decisions, Incorporated, has been conduction a “Quality of Life Survey and has been 
issuing the “MDI Metro Milwaukee Quality of Life Report” in 1994, 1995 and 1996. The following 
information is directly from the “1996 MATC Student Needs Report” prepared by Management 
Decisions, Incorporated for MATC in September of 1 996. 

“The image of MATC and of 49 other metro organizations as quality of life contributors were evaluated in 
two ways on the 1996 MDI Metro Quality of Life Report: 

* Their average score on a 1 1 point scale among only those who are aware of and feel they can rate an 
organization and its contribution; and 

* the percentage who give a positive or high rating among all adults, both those who are aware and those 
unaware of an organization. These two evaluation scores are highly interrelated. 

Overall, ratings for Metro Milwaukee organizations declined between 1994 and 1995, only to increase 
again over the past year. In 1994, the average rating scores for the 44 metro organizations were 7.04 and 
43%, respectively. In 1995, the average rating scores for 51 metro organizations were 6.63 and 33%. By 
1996, the average rating scores of 50 metro organizations are 6.90 and 38%. 

Rating scores are likely to reflect both the perceived quality (“what is contributed ) and the quantity ( how 
much is contributed”) to the area, as well as the success of organizations in communicating such 
information to the public. 

Milwaukee Area Technical College has remained among the top contributors to the area’s quality of life 
for the past three years. Over the past two years, the college has been in the top ten. Milwaukee Public 



Television. WMVS/WMVT-TV, is affiliated with MATC and remains among the top five area 
contributors. 



Table 5 The top organizations as rated by Metro Milwa 


ukee residents: 




1994 


1995 


1996 


1994 


1995 


1996 


1 . 


Pettit Foundation 


8.59 


8.47 


8.40 


68% 


68% 


70% 


2. 


Milwaukee World Festival 


7.95 


7.86 


8.30 


66% 


62% 


74% 


3. 


Harley Davidson 


8.15 


7.77 


8.06 


68% 


60% 


66% 


4. 


Bradley Foundation 


7.88 


7.78 


8.05 


57% 


49% 


59% 


5. 


Milwaukee Public Television 


8.00 


7.91 


8.00 


66% 


62% 


64% 


6. 


Medical College of Wisconsin 


7.75 


7.52 


7.69 


52% 


46% 


50% 


7. 


Milwaukee Area Technical College 


7.36 


7.50 


7.58 


49% 


51% 


55% 


8. 


Midwest Express Airlines 


7.12 


7.33 


7.57 


44% 


43% 


50% 


9. 


Miller Brewing 


7.58 


7.25 


7.48 


56% 


46% 


53% 


10. 


University of Wisconsin-Milwaukee 


7.27 


7.31 


7.47 


50% 


48% 


54% 



The report goes on to say; 

“Furthermore, MATC is on of the few organizations whose ratings have increased steadily in each of the 
past three years. Additionally, the school receives a superlative rating of “10” from one-fifth of all 
residents and an extremely low share of poor ratings - 3%.” 



Handsworth College’s value to its local community is represented by its rapid growth over the past few 
years as shown by the numbers of students enrolling each year. 



Table 6: Handsworth College Enrollments 1990 - 97 


Year 


Number of 


% Growth 




Enrollments 




1990/91 


5,442 




1991/92 


7,018 


29.0% 


1992/93 


9,650 


37.5% 


1993/94 


15,063 


56.1% 


1994/95 


19,193 


27.4% 


1995/96 


22,372 


11.6% 


1996/97 


24,000 (est.) 


10.7% 



The rapid growth in enrollments between 1991 and 1995 is mostly due to Handsworth College becoming 
more closely involved with local community groups. 

Many of the population in the area immediately surrounding the college are of Indian, Pakistani and Afro- 
Caribbean origin who have had limited integration with the rest of the population, and are unlikely to 
enroll at public colleges. However, a considerable amount of cultural education did take place within 
community groups and at religious centers. 
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Handsworth College has pioneered schemes that enabled these groups to obtain national accreditation for 
the study that takes place. The college works in close partnership with them in developing the curriculum 
and has been able to secure government (FEFC) funding to enable the work to develop. The college 
provides teacher training to the tutors and offers other support and guidance to ensure that the groups 
operate effectively and efficiently. 



Following this success of these activities, the schemes have been expanded to include other community 
groups such as Chinese and Vietnamese and groups outside the immediate local area. Furthermore, the 
college’s expertise and good infrastructure has allowed the model to expand nationally and now groups all 
over the country are being supported by the college. 

In the last two years, the numbers have become stabilized as the provision has become consolidated into 
systematic college activity. 



Comparative Results for our Colleges 

As is shown in tables 3 and 4, the indicators used at Handsw-orth are much more biased towards the 
financial health of the college than those at Milwaukee. This is because funding follows the performance 
indicators in the UK and therefore will be more likely to address financial issues. These financial issues 
are addressed in the US via annual independent financial audits. 

The calculation below' shows a comparison for the income received for each full-time equivalent student in 
each college in 1995/96. 



Table 7 


Funding and FTE Student Numbers 1995/96 




Milwaukee 


Handsworth 


Ratio 


Operational Expenditure 


$113,165,000 


$20,899,000 




FTE students 


12,307 


6,492 




Expenditure per FTE 


$9,195 


$3,219 


2.86:1 


Gross Domestic Product (GDP) 
per head of population 


$29,600 


$20,900 




Expenditure per FTE as a 
percentage of GDP per head 


31.1% 


15.4% 


2.02:1 




Hence, in Milwaukee, further education is funded at a much higher rate than Handsworth, and even after 
taking relative costs into account, the rate is double. 



Table 8 shows the costings based on a head-count of students rather than FTEs. 



Tabic 8 Funding and Student Headcount numbers 1995/96 




Milwaukee 


Handsworth 


Ratio 


Student headcount 


63,270 


22,372 






Expenditure per student 


$1,789 


$934 


1.92:1 




Expenditure per student as a 
percentage of GDP per head 


6.04% 


4.47% 


1.35:1 





(It must be stated that Handsworth College is poorly funded even by UK standards as it has the lowest 
Average Level of Funding in England. The average expenditure per FTE in further education in England 
in 1995/96 was $4,435. Handsworth fares so badly due to historical factors resulting from low levels of 
local funding prior to the FEFC being established in 1993 and also by the way the funding methodology 
handled the college’s large growth in 1993/94. However, the FEFC's policy of 'convergence' should 
increase Handsworth College’s ALF.) 



It is useful to attempt to compare the graduation rates of each of the two colleges. 

In 1995/96, MATC had a graduation rate of 36% compared to the state average for Wisconsin of 43%. In 
the most recently published figures for 1994/95, Handsworth College was reported as having an 
achievement rate of 92% compared to the average for England of 75%. However, these comparisons are 
grossly misleading as the published English figures are based on the achievement rate of completing 
students (and not enrollments) and also include qualifications of all types. This particularly distorts the 
result for Handsworth College where a large numbers of short programs are delivered, normally with high 
achievement rates. 

It is better to compare the broadly similar programs using US terminology of Associate Degrees, Two- 
Year Diplomas, One-Year Diplomas and Short Programs. These are calculated by comparing the 
numbers graduating in 1995/96 against those enrolling as shown below: 



Table 9 


Graduation Rates 1995/96 


Type of Program 


Milwaukee 


Handsworth 


Associate Degrees 


39% 


n/a 


Two-Year Diplomas 


32% 


32% 


One-Year Diplomas 


37% 


46% 


Short Programs 


n/a 


79% 
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In this study, we have not been able to compare the depth and breadth of the academic content in the 
programs in the two colleges. However, Handsworth College is about to commence a program of 
Associate Degrees recognized in the US, where it was found that the depth of the existing Handsworth 
College programs were very similar to those in the US but additional study of about 25-30% of the 
program needs to be added to give sufficient breadth. 



Conclusions 

In both colleges, hard practical reality decrees that the principal use of performance indicators contribute 
towards determining each colleges’ annual funding budget. In Milwaukee, the student FTE number is 
used but the college’s budget depends to a great extent on factors outside the college’s control. 

At Handsworth, the funding received from the FEFC is based on pre-defined criteria which involve the 
following:: 

* the ratio of the actual volume of education delivered compared to target 

* the college’s average level of funding 

* the college’s contribution to the FEFC’s main priorities, which up until now has been growth 

Student retention rates and graduation rates and integral features of the calculation of the volume of 
education delivered and qualifications which contribute to the national targets (NTETS) have higher 
funding unit values. Hence the college funding is related to good scores in the FEFC’s performance 
indicators PI 1, PI 3, PI 4, PI 5 and PI 6 that were defined earlier in this paper. In addition, colleges who 
receive poor inspection gradings are not allowed to grow in the areas that are criticized. 

Milwaukee is apparently substantially better funded than Handsworth. Using the most appropriate 
indicator which is college’s income per FTE student as a percentage of the GDP per head of population, 
Milwaukee is better funded by a ratio of two to one. There are several reasons, for this. The programs of 
study at Milwaukee have a greater breadth which probably requires greater resources to deliver. The 
unemployment rate in Handsworth (and Birmingham and the UK) is much higher than Milwaukee 
meaning that the requirement for training employed students will be lower. Generally the student 
participation rate at Handsworth is lower, suggesting that the population local to Handsworth places less 
emphasis on further education than does Milwaukee. (This is probably also true if the UK were to be 
compared to the US) 

The graduation rates of both colleges are broadly similar suggesting that the colleges’ educational 
standards are comparable. 

Both colleges have a close involvement with and add value to their local communities. These cannot be 
compared directly, but Milwaukee’s ‘Quality of Life’ shows the level of satisfaction in the local 
community as does the large growth in Handsworth’s working with local community' groups. 

Handsworth differs from Milwaukee in that there is strong local competition from other colleges. In 
Birmingham there are eight general further education colleges who recruit from the same potential 
student base. Each college runs aggressive marketing campaigns to try to increase its student numbers. 

In many large cities in England colleges have recently merged and this may yet happen in Birmingham. 
Milwaukee Area Technical College, at this time, has little competition from publicly funded colleges. 

The further education colleges in the UK are firmly attached to funding driven by performance indicators. 
The history and lessons that are being learned in the UK should be examined by educators from other 
countries, such as the US, that may be moving toward performance based funding. A fair warning to 
educators on the US side of the Atlantic was given by Austin R. Gilbert, chainnan of South Carolina s 
Commission on Higher Education: “We are faced with a crisis in credibility'.” If South Carolina’s public 



colleges want more state money, he said, they must submit to high-stakes performance evaluations and 
demonstrate that they have improved. Educators in all states should be watching this closely. 



There are several issues that manifest themselves in regard to performance indicators. These have to do 
with the questions; What are we measuring? And why are we measuring it? If we are measuring our 
outcomes, what are they? If we are measuring quality, what is quality? Why are we measuring these 
things? There appears to be two justifications arising for developing performance indicators. One is for 
funding purposes and the other is for program improvement. Currently, the funding objective is more 
evident and better defined and implemented in the UK than in the US. Program improvement, on the 
other hand, is an often stated objective of performance indicators in the US, but the link between data 
gathering and actual program improvement is most often mired in complex political processes both in and 
outside of the various colleges. This is further complicated by the fact that many performance indicators 
are not indicators of performance at all but rather management or accounting tools. The danger is that 
there may be a growing distinction between what we do and how we look. What we do in further 
education will probably continue to respond in some way to local markets while how we look, that is how 
we portray ourselves to outside agencies, will be determined by the unique subsets of information they may 
require for funding or for accreditation. 

Our challenge will be to plan and deliver what is best for our students (customers) within the context of 
outside funding and accreditation criteria (performance indicators). In recent years there have been 
discussions of quality in education largely as the result of business practices and terminology that have 
migrated into education. In many places “quality” has replaced “excellence” as the catch word for the 
nature of the goal of our endeavor. For our purposes we will use the term “excellence” to mean either 
quality or excellence. 

What has been traditionally measured in the assessment of education have been profiles of the colleges. 
Such things as library holdings, academic preparation of faculty, resources available per student have been 
used to determine accreditation and rank. These indicators are not outcome indicators but may still be 
considered as performance indicators within the following discussion. There are two kinds of 
indicators that help us define and manage our colleges and subsequently portray our colleges to others. 

The first, are prerequisites to excellence. These are the above mentioned traditional indicators. They 
are the kinds of things most of us agree will be needed for the college to create and environment in which 
excellence can occur. The second indicators are evidences of excellence. These are the outcomes that 
tell us that we are doing well. In a general way, the prerequisites to excellence are the old indicators and 
the evidences of excellence are the new indicators. 

Our task is to maintain the integrity of our institutions relative to their missions while at the same time 
responding to these outside forces. It seems that most often these outside indicators are thrust upon us 
and our task becomes trying to determine how these indicators can help us better understand and manage 
our institutions. The indicators themselves seldom grow out of our wish to better understand and manage 
our institutions and therefore may not be well suited for that activity. 

Funding, on the other hand, poses problems because it is attempting to emulate the free market using the 
indicators as a proxy for the market place. While the free market players are organized to earn money the 
public sector service providers are organized to spend money. The mismatch is obvious. In addition the 
messages inherent in this activity are garbled. This is further complicated by the different nature and 
orientation of the training of chief financial officers and chief academic officers. As long as performance 
indicators are tied to funding we will not use them to manage the college we will use them to manage the 
budget. 



Discussion 
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Suggestions for Further Study 



In any study, the more the authors explore the topics, the more the possibilities for further work appear, 
often at a much greater rate than the authors can handle. This is particularly the case when the authors 
are based 4,500 miles apart and also have to manage the many differences in concepts and terminology 
that exist in the educational systems between our countries.. 

This study could be extended in several ways that include: 

* consider the numerical values of other performance indicators listed in this study 

* include more colleges in the US, UK and other countries 

* consider the issues of quality and excellence in greater depth. 



AACC 


Glossary of Terms 

American Association of Community Colleges 


ALF 


Average level of funding (UK) 


El 


Equalization index (US) 


ESL 


English as a second language 


FE 


Further education 


FEFC 


Further Education Funding Council (UK) 


FTE 


Full time equivalent (US and UK) 


GDP 


Gross Domestic Product 


MATC 


Milwaukee Area Technical College 


MDI 


Management Decisions Incorporated (US) 


NTETS 


National Training and Education Targets (UK) 
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